Pre-train, Interact, Fine-tune: A Novel Interaction Representation for Text Classification

Text representation can aid machines in understanding text. Previous work on text representation often focuses on the so-called forward implication, i.e., preceding words are taken as the context of later words for creating representations, thus ignoring the fact that the semantics of a text segment is a product of the mutual implication of words in the text: later words contribute to the meaning of preceding words. We introduce the concept of interaction and propose a two-perspective interaction representation, that encapsulates a local and a global interaction representation. Here, a local interaction representation is one that interacts among words with parent-children relationships on the syntactic trees and a global interaction interpretation is one that interacts among all the words in a sentence. We combine the two interaction representations to develop a Hybrid Interaction Representation (HIR). Inspired by existing feature-based and fine-tuning-based pretrain-finetuning approaches to language models, we integrate the advantages of feature-based and fine-tuning-based methods to propose the Pre-train, Interact, Fine-tune (PIF) architecture. We evaluate our proposed models on five widely-used datasets for text classification tasks. Our ensemble method, outperforms state-of-the-art baselines with improvements ranging from 2.03% to 3.15% in terms of error rate. In addition, we find that, the improvements of PIF against most state-of-the-art methods is not affected by increasing of the length of the text.Comment: 32 pages, 5 figure

MsPrompt: Multi-step Prompt Learning for Debiasing Few-shot Event Detection

Event detection (ED) is aimed to identify the key trigger words in unstructured text and predict the event types accordingly. Traditional ED models are too data-hungry to accommodate real applications with scarce labeled data. Besides, typical ED models are facing the context-bypassing and disabled generalization issues caused by the trigger bias stemming from ED datasets. Therefore, we focus on the true few-shot paradigm to satisfy the low-resource scenarios. In particular, we propose a multi-step prompt learning model (MsPrompt) for debiasing few-shot event detection, that consists of the following three components: an under-sampling module targeting to construct a novel training set that accommodates the true few-shot setting, a multi-step prompt module equipped with a knowledge-enhanced ontology to leverage the event semantics and latent prior knowledge in the PLMs sufficiently for tackling the context-bypassing problem, and a prototypical module compensating for the weakness of classifying events with sparse data and boost the generalization performance. Experiments on two public datasets ACE-2005 and FewEvent show that MsPrompt can outperform the state-of-the-art models, especially in the strict low-resource scenarios reporting 11.43% improvement in terms of weighted F1-score against the best-performing baseline and achieving an outstanding debiasing performance

Diffusion-Augmented Depth Prediction with Sparse Annotations

Depth estimation aims to predict dense depth maps. In autonomous driving scenes, sparsity of annotations makes the task challenging. Supervised models produce concave objects due to insufficient structural information. They overfit to valid pixels and fail to restore spatial structures. Self-supervised methods are proposed for the problem. Their robustness is limited by pose estimation, leading to erroneous results in natural scenes. In this paper, we propose a supervised framework termed Diffusion-Augmented Depth Prediction (DADP). We leverage the structural characteristics of diffusion model to enforce depth structures of depth models in a plug-and-play manner. An object-guided integrality loss is also proposed to further enhance regional structure integrality by fetching objective information. We evaluate DADP on three driving benchmarks and achieve significant improvements in depth structures and robustness. Our work provides a new perspective on depth estimation with sparse annotations in autonomous driving scenes.Comment: Accepted by ACM MM'202

Effects of typhoons on surface seawater pCO(2) and air-sea CO2 fluxes in the Northern South China Sea

Author Posting. © American Geophysical Union, 2020. This article is posted here by permission of American Geophysical Union for personal use, not for redistribution. The definitive version was published in Journal of Geophysical Research: Oceans 125(8), (2020): e2020JC016258, doi:10.1029/2020JC016258.This study assessed the effects of typhoons on sea surface pCO2 and CO2 flux in the northern South China Sea (SCS). During the passage of three major typhoons from May to August 2013, sea surface pCO2, surface seawater temperature (SST), and other meteorological parameters were continuously measured on a moored buoy. Surface water in the region was a source of CO2 to the atmosphere with large variations ranging from hours to months. SST was the primary factor controlling the variation of surface pCO2 through most of the time period. Typhoons are seen to impact surface pCO2 in three steps: first by cooling, thus decreasing surface pCO2, and then by causing vertical mixing that brings up deep, high‐CO2 water, and lastly triggering net uptake of CO2 due to the nutrients brought up in this deep water. The typhoons of this study primarily impacted air‐sea CO2 flux via increasing wind speeds. The mean CO2 flux during a typhoon ranged from 3.6 to 5.4 times the pretyphoon mean flux. The magnitude of the CO2 flux during typhoons was strongly inversely correlated with the typhoon center distance. The effect of typhoons accounted for 22% of the total CO2 flux in the study period, during which typhoons occurred only 9% of the time. It was estimated that typhoons enhanced annual CO2 efflux by 23–56% in the northern SCS during the last decade. As such, tropical cyclones may play a large and increasingly important role in controlling CO2 fluxes in a warmer and stormier ocean of the future.This study was supported by the Marine Public Welfare Project of China (Grant 200905012), the Scientific Research Fund of the Second Institute of Oceanography of China (Grant JT1502), the Global Change and Air‐Sea Interaction project of China (Grant GASI‐03‐01‐02‐02), and the National Natural Sciences Foundation of China (Grant 91128212).2021-02-0